Skip to content

387 audio dubbing feature#554

Open
LeviXIII wants to merge 58 commits intomainfrom
387-audio-dubbing-feature
Open

387 audio dubbing feature#554
LeviXIII wants to merge 58 commits intomainfrom
387-audio-dubbing-feature

Conversation

@LeviXIII
Copy link
Contributor

@LeviXIII LeviXIII commented Jan 30, 2026

IMPORTANT NOTE:

  • You will need to run pnpm install in the codex/webviews folder
  • After that go back to the top level codex folder and run pnpm run build:webviews
    If you do not do this, the UI will be messed up.

Video Playback Enhancements

  • Upgraded React Player to v3.4.0 and fixed compatibility issues.
  • Added support for .webm video playback.
  • Fixed multiple bugs where videos:
    • Would not play when clicking Show Video.
    • Failed to autoplay.
    • Would not resume correctly after audio playback.
  • Improved video readiness checks to prevent race condition errors.
  • Stabilized playback to eliminate play / pause loops.

Audio Playback & Synchronization

  • Improved audio + video synchronization, so recorded audio now plays at correct timestamps.
  • Fixed overlapping audio issues by:
    • Merging audio into a single blob instead of playing multiple audio elements.

Ensured:

  • Audio pauses video when finished.
  • Audio and video cannot fight for playback control.
  • Added safeguards to prevent crashes when switching audio quickly.
  • Added audio warnings when no audio exists.

Timestamp Editing & UI

  • Added full audio timestamp editing system:
    • Current cell
    • Previous cell
    • Next cell
  • Added:
    • Timestamp sliders
    • Countdown button
    • Target duration bar under waveform
    • Loading spinner on play buttons
    • Enabled overlapping audio + subtitles across adjacent cells.
    • Improved timestamp initialization when new audio is recorded.
    • Updated UI in the Timestamps tab to be clearer and easier to use.

File Saving & Project Structure Fixes

  • Fixed video file saving from the Edit Metadata modal.
  • Standardized file output so all files now save correctly into:
    • .project/pointer
    • .project/files

Exporting

  • Allows for exporting cue split timestamps
  • Alerts the user before exporting if there are timestamps that overlap in their file
  • Fixes audio exporting file naming convention

Code Quality & Stability

  • Cleaned up:
    • Dead code
    • Unused imports
    • Duplicate components
    • Improved type safety across audio + video playback handling.
    • Refactored player references for better maintainability.

Added unit and integration tests for audio synchronization logic.

- Files now save to pointer and file folders in the .project folder regardless of where they currently reside.
…itor.

- Introduce CSS injection plugin in Vite config for better styling management specifically with the new version of react-player.
- Adjust player references to accommodate new methods and ensure compatibility with updated video functionalities.
- Comment out VideoTimelineEditor for now.
- Video player prop adjustments.
…ion with audio.

- Implement video readiness checks and improve audio URL handling to ensure seamless playback experience stopping errors from race conditions.
- Consolidate ReactPlayerRef interface to a type folder.
…o overtop the video for the current file.

- Recorded audio adjusts based on timestamp.
…y's as source of truth and extracted it to the old AudioPlayButton.
- Add type safety for contentRef and streamline audio playback handling.
…tab to be able to overlap audio and text.

- Suppression of some warnings and errors as they will come if there is no audio to overlap in the previous or next cells.
- Add timestamps to empty cells.
…aying and vice versa.

- Allow for GlobalMessage object to receive full object (including destination used for audioStateChange events)
…nd next timestamp information for subtitle files.
- Implemented new message handlers for updating and requesting audio timestamps in the CodexCellEditor.
- Added functionality to update audio timestamps in the document model
- Enhanced the UI to display audio timestamp sliders for current, previous, and next cells.
- Refactored cell conversion logic to include audio timestamps.
- Updated types to support audio timestamp data in editor messages and cell content.
- Improved audio playback management to synchronize with video playback and handle audio state effectively.
…ead of trying to play 3 different audio elements at once.
- Added functionality to refresh audio timestamps for adjacent cells when their audio attachment state changes.
- Implemented initialization of audio timestamps when the Timestamps tab is opened with newly recorded audio.
- Improved handling of audio timestamps to ensure synchronization across cells.
@LeviXIII LeviXIII linked an issue Jan 30, 2026 that may be closed by this pull request
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Audio Dubbing feature

1 participant